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Report  No.  D-2001-179  September  10,  2001 

(Project  No.  D2000CB-0236.000) 

Military  Aircraft  Accident  Investigation  and  Reporting 
Executive  Summary 


Introduction.  This  evaluation  was  initiated  in  response  to  a  request  from  the  Deputy 
Under  Seeretary  of  Defense  (Installations  and  Environment),  to  eonduet  a  proeess  review 
of  the  Military  Serviee’s  implementation  of  the  DoD  poliey  as  stated  in  DoD  Instruetion 
6055.7,  “Mishap  Investigation,  Reporting,  and  Reeordkeeping,”  April  10,  1989. 

The  Deputy  Under  Seeretary  of  Defense  (Installations  and  Environment)  asserted  that 
improving  aviation  safety  was  a  major  objeetive  of  DoD.  Military  Department  aviation 
safety  programs  foeused  on  the  reduetion  and  prevention  of  aviation  mishaps;  however, 
aviation  mishaps  eontinued  to  injure  people  and  eaused  property  damage  and  losses. 
During  EY  20()0,  the  Serviees  experienced  a  total  of  57  class  “A”  aircraft  mishaps,  which 
resulted  in  58  deaths  and  44  aircraft  destroyed.  A  class  “A”  mishap  occurred  when  the 
reportable  damage  was  $1  million  or  more;  there  was  total  destruction  of  the  aircraft;  or 
an  injury  resulted  in  a  fatality  or  permanent  total  disability. 

Objectives.  Our  overall  objective  was  to  determine  whether  the  Services  had 
implemented  DoD  policy  for  aircraft  mishap  investigations  and  reporting.  Specifically, 
we  evaluated: 

•  the  independence  of  mishap  investigation  boards, 

•  the  root  cause  identification  process, 

•  the  timeliness  and  factual  accuracy  of  investigation  reports, 

•  the  recordkeeping  of  mishaps,  and 

•  the  process  for  tracking  corrective  actions. 

The  evaluation  was  a  process  review  and  did  not  verify  the  implementation  of  appropriate 
corrective  actions.  We  also  reviewed  the  management  control  program  as  it  applied  to 
the  evaluation  objectives. 

Results.  The  Military  Departments  implemented  the  requirements  in  DoD 
Instruction  6055.7  through  their  respective  regulations  and  instructions.  Each  Military 
Department's  aircraft  mishap  investigation  board  was  independent  and  identified  root 
causes  in  its  mishap  reports,  which  were  timely. 

The  Military  Departments  generally  implemented  DoD  policy  for  aircraft  mishap 
investigations,  reports,  and  recordkeeping  effectively,  but  one  issue  merits  management 
attention.  Although  the  Army  and  the  Air  Eorce  processes  for  tracking  the  status  of  open 
corrective  actions  were  adequate,  the  Navy  process  was  not  effective.  Consequently,  the 
corrective  actions  data  in  the  Navy  Safety  Information  Management  System  were 


unreliable.  Also,  the  Naval  Safety  Center  was  unable  to  meet  the  strategie  mission  for 
making  safety  information  easily  aeeessible  to  all  Navy  personnel  and  to  efficiently 
report  corrective  action  performance. 

See  Appendix  A  for  details  on  the  management  control  program. 

Summary  of  Recommendations.  We  recommend  that  the  Vice  Chief  of  Naval 
Operations  direct  the  Naval  Safety  Center  to  update  the  safety  information  management 
system  to  track  the  timely  status  of  corrective  actions  associated  with  aircraft  mishaps 
and  simplify  system  access  for  management  and  safety  officials. 

Management  Comments.  The  Vice  Chief  of  Naval  Operations  concurred  with  the 
recommendations  and  stated  that  the  Navy  recognizes  the  weakness  in  corrective-action 
tracking.  The  Navy  also  agreed  that  the  recordkeeping  process  was  not  effective.  The 
Naval  Safety  Center  has  taken  or  is  planning  to  take  actions  to  improve  corrective-action 
tracking  by  increasing  personnel  and  forwarding  lists  of  open  recommendations  to 
appropriate  agencies  for  action,  controlling  custodians,  and  command  aircraft  analysts. 
The  Vice  Chief  of  Naval  Operations  stated  that  two  initiatives,  Navy/Marine  Corps 
Intranet  and  Web  Enabled  Navy,  will  establish  a  conduit  for  fully  functional  Internet 
access  to  Naval  Safety  Center  databases  for  authorized  customers  and  limited  access  to 
non-privileged  data  for  all  others.  A  discussion  of  the  management  comments  is  in  the 
Finding  section  of  the  report,  and  the  complete  text  is  in  the  Management  Comments 
section. 
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Background 


Federal  Aviation  Safety  Programs.  The  Code  of  Federal  Regulations,  Title  41, 
subpart  101-37.12,  “Federal  Ageney  Aviation  Safety  Program,”  July  1,  1999, 
reeommends  the  elements  for  Federal  aviation  safety  programs.  The  elements 
include; 

•  an  aviation  safety  council, 

•  inspections  and  evaluations, 

•  hazard  reporting, 

•  aircraft  mishap  and  incident  investigations, 

•  education  and  training,  and 

•  aviation  qualification  and  certification. 

The  DoD  aviation  safety  program  generally  contained  all  of  the  recommended 
elements. 

DoD  Aviation  Safety,  Safety  and  the  achievement  of  low  mishap  rates  were 
integral  parts  of  the  readiness  focus  for  FY  2000.  The  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  requested  that,  for  FY  2000, 
the  Senior  Readiness  Oversight  Council  concentrate  on  updating  the  Military 
Department’s  Aviation  Safety  Program. 

During  FY  2000,  the  Military  Services  experienced  a  total  of  57  class  “A”  aircraft 
mishaps,  which  resulted  in  58  deaths  and  44  aircraft  destroyed.  A  class  “A” 
mishap  occurred  when  the  reportable  damage  was  $1  million  or  more;  the  aircraft 
was  destroyed;  or  an  injury  resulted  in  a  fatality  or  permanent  total  disability. 

Mishap  Investigation,  Reporting,  and  Recordkeeping.  The  Deputy  Under 
Secretary  of  Defense  (Installations  and  Environment)  requested  that  the  Inspector 
General,  DoD,  conduct  a  process  review  of  the  Service’s  implementation  of  DoD 
policy  for  aircraft  accident  investigation  and  reporting.  This  policy  is  stated  in 
DoD  Instruction  (DoDI)  6055.7,  “Mishap  Investigation,  Reporting,  and 
Recordkeeping,”  April  10,  1989. 

DoDI  6055.7  requires  the  Heads  of  DoD  Components  to: 

•  develop  qualification  criteria  for  mishap  investigators,  provide  report 
reviews,  and  record  collected  data, 

•  adopt  mishap  categories,  classification  criteria,  and  reporting  formats 
and  procedures  contained  in  the  instruction,  and 

•  ensure  identification  and  corrective  actions  on  mishap  causal  factors. 
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DoD  classified  mishaps  according  to  the  severity  of  the  injury  or  property 
damage  using  designated  letters  of  the  alphabet.  DoDI  6055.7  based  the 
reporting  format  and  procedures  on  the  type  of  investigations  performed, 
specifically  safety  and  legal  investigations. 

Safety  and  Legal  Investigation  Processes.  DoD  used  two  distinct  investigation 
processes  for  serious  mishaps,  safety  and  legal.  The  sole  objective  of  DoD  safety 
investigations  was  to  prevent  future  mishaps.  Safety  teams  conducted 
investigations  and  determined  root  eauses  to  make  improvements  in  aviation 
safety  and  prevent  aviation  mishaps.  A  key  element  to  aviation  safety  was  mishap 
reporting.  Mishap  reporting  focused  on  investigation  results  and  reeommended 
correetion  of  causes  determined  by  safety  investigation  teams.  Safety  teams 
forwarded  final  reports  through  an  endorsement  chain  for  acceptance  or  rejeetion 
of  reeommended  eorreetive  actions.  Key  program  managers  were  provided  with 
information  to  implement  reeommended  corrective  actions  in  areas  like: 
supervising,  training,  logistics,  or  maintenance  of  aviation  programs.  Also, 
Military  Departments  needed  to  eonsider  and  use  historical  safety  data  during  the 
development  and  aequisition  of  new  systems. 

Legal  investigations  were  conducted  simultaneously,  but  separately  from  the 
safety  investigations.  Legal  investigations  foeused  on  preserving  the  evidence  of 
a  mishap  for  all  purposes  other  than  mishap  prevention.  When  legal 
investigations  were  completed,  a  publicly  releasable  report  was  produced  that 
fully  doeumented  the  facts,  circumstances,  and  causes  of  a  mishap.  The 
appointed  offieers  condueted  investigations,  determined  findings,  and  made 
recommendations.  The  investigators  consulted  legal  advisors,  who  were  made 
available  by  the  Serviees'  judge  advoeate  generals.  Once  the  legal  review  was 
completed,  the  reports  were  forwarded  to  the  commander  of  the  mishap  unit  to 
assign  responsibility  and  aecountability. 

Role  of  the  Military  Department  Safety  Centers.  The  Military  Departments 
tasked  the  safety  centers  to: 

•  establish  eriteria  for  reporting  and  quality  control  of  mishap  data, 

•  develop  a  responsive  and  aceessible  safety  database, 

•  analyze  mishap  eausal  factors,  systemic  origins,  and  trends,  and 

•  ineorporate  proeedures  for  developing  mishap  prevention  programs; 
and  track  recommended  eorreetive  actions. 

See  Appendix  B  for  a  diseussion  of  the  eorreetive  aetions  closure  proeess.  As 
Military  Department  safety  ehiefs,  the  commanders  of  the  safety  eenters 
periodically  advised  their  Chiefs  of  Staff  about  aviation  safety  matters  (for 
example,  information  on  the  details  of  a  mishap,  or  the  status  of  eorreetive 
actions). 
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Objectives 


Our  overall  objective  was  to  determine  whether  the  Services  have  implemented 
DoD  policy  for  aircraft  mishap  investigation  and  reporting.  Specifically,  we 
evaluated: 

•  the  independence  of  mishap  investigation  boards, 

•  the  root  cause  identification  process, 

•  the  timeliness  and  factual  accuracy  of  investigation  reports, 

•  the  recordkeeping  of  mishaps,  and 

•  the  process  for  tracking  corrective  actions. 

The  evaluation  was  a  process  review  and  did  not  verify  the  implementation  of 
appropriate  corrective  actions.  See  Appendix  A  for  details  on  the  management 
control  program. 
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Aircraft  Mishap  Recordkeeping  Process 

The  Military  Departments  generally  implemented  DoD  policy  for  aircraft 
mishap  investigations,  reports,  and  recordkeeping  effectively,  but  one 
issue  merits  management  attention.  Although  the  Army  and  the  Air  Force 
processes  for  tracking  the  status  of  open  corrective  actions  were  adequate, 
the  Navy  process  was  not  effective.  The  recordkeeping  process  was  not 
effective  because  the  Naval  Safety  Center  (NSC)  set  a  low  priority  for 
updating  the  status  of  corrective  actions  data  in  the  Safety  Information 
Management  System  (SIMS)  and  had  a  significant  data  backlog.  Also,  the 
SIMS  database  design  required  advanced  systems  language  literacy  for 
querying.  Consequently,  the  status  of  corrective  actions  in  SIMS  was 
unreliable.  Also,  the  NSC  was  unable  to  meet  their  strategic  mission  for 
making  safety  information  easily  accessible  to  all  Navy  personnel  or  to 
efficiently  report  corrective  action  performance. 

Mishap  Investigations  and  Reports 


Military  Department  Guidance.  The  Military  Department  generally 
implemented  the  DoD  policy  for  aircraft  mishap  investigations,  reports,  and 
recordkeeping  effectively.  DoDI  6055.7  requires  that  the  Military  Departments 
establish  procedures  to  collect,  maintain,  analyze,  and  report  aircraft  mishaps. 
Each  Military  Department  establishes  regulations  or  instructions  for  aviation 
mishap/accident  investigations,  reports,  and  recordkeeping. 

•  Army  Regulation  385-40,  “Accident  Reporting  and  Records,” 
November  1,1994. 

•  Office  of  the  Chief  of  Naval  Operations  Instruction  3750. 6Q,  “The 
Naval  Aviation  Safety  Program,”  August  28,1989,  establishes  Navy 
policy.  The  United  States  Marine  Corps  follows  the  guidance 
provided  by  the  Navy  for  implementing  aviation  mishap  policy  and 
procedures. 

•  Air  Force  Instruction  91-204,  “Safety  Investigations  and  Reports,” 
November  29,  1999. 

Guidance  for  Investigations.  The  Military  Department  guidance  directs  board 
members  and  a  disinterested  third  party  to  conduct  class  “A”  aircraft  mishap 
investigations.  Safety  investigation  boards  included  senior  personnel  from 
outside  the  mishap  unit  and  board  members  with  experience,  training,  and 
knowledge  in  related  specialized  fields.  Safety  investigation  boards  followed  the 
guidance  outlined  by  each  Military  Department.  Legal  investigations,  conducted 
by  one  or  more  individuals,  followed  guidance  from  the  offices  of  the  judge 
advocate  generals,  legal  counsel,  and  other  authorities.  Mishap  board  members 
assigned  to  conduct  safety  investigations  did  not  conduct  legal  investigations  of 
the  same  mishap. 
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The  Military  Departments  also  established  struetured  data  eolleetion  proeedures 
for  data  analyses.  The  data  analyses  were  used  to  identify  eausal  factors 
associated  with  aircraft  mishaps.  Safety  and  legal  investigators  completed  a 
thorough  documentation  process  before  identifying  causal  factors  and 
recommending  corrective  actions  (safety)  or  determining  responsibility  (legal). 
While  reviewing  14  legal  reports,  we  determined  that  the  information  generally 
matched  the  causes  identified  in  the  safety  investigations. 

Guidance  for  Reporting,  The  Services'  safety  investigation  guidance  requires  an 
initial  report  30  days  after  a  class  “A”  mishap  occurs,  for  endorsement  through  the 
chain  of  command.  Each  of  the  Services  generally  released  an  initial  safety 
investigation  report  30  to  90  days  after  the  aircraft  mishap.  The  endorsers 
commented  on  factual  accuracy,  recommendations,  and  assignments  for 
corrective  actions  in  the  safety  investigation  report.  According  to  legal  guidance, 
the  convening  authority  determined  investigation  timelines.  The  judge  advocate 
generals  reviewed  legal  reports  for  legal  compliance  and  the  convening  authority 
reviewed  investigation  results  for  action. 

Recordkeeping  and  Corrective  Actions  Tracking 


Database  Systems  for  Recordkeeping,  The  Military  Departments  designated 
the  safety  centers  as  the  administrators  for  historical  data  on  hazards  and  mishaps. 
The  Army  and  Air  Force  database  systems  were  available  through  the  internet  to 
authorized  personnel  worldwide,  and  permitted  direct  database  querying  and  data 
entry.  Database  updates  occurred  daily  to  maintain  information  for  data  analyses. 
Information  from  the  database  systems  provided  timely  data  to  management  and 
field  organizations  to  forecast  the  highest  risk  operations  and  recommend  controls 
to  reduce  risk. 

Tracking  and  Closing  Corrective  Action,  The  Military  Departments  used 
different  processes  for  tracking  and  closing  recommended  corrective  actions.  The 
Military  Departments  had  591  open  corrective  actions  for  FYs  1995  through  2000 
for  class  “A”  aircraft  mishaps.  See  Appendix  B  for  details  on  the  number  of  open 
corrective  actions. 

Army  and  Air  Force  Process 


Both  Army  and  Air  Force  have  enhanced  recordkeeping  systems.  The  systems 
are  web-based  and  offer  a  user-friendly  query  process  available  to  commands  in 
remote  locations  and  both  Services  have  provided  training  and  on-line  user  aids. 
Army  and  Air  Force  Safety  Centers  maintained  the  tracking  databases  in  a  timely 
manner.  The  Air  Force  shows  a  higher  number  of  unresolved  corrective  actions 
because  it  does  not  close  an  action  until  implementation  is  complete.  Both  Army 
and  Air  Force  Chiefs  of  Staff  are  briefed  regularly  on  the  status  of  unresolved 
corrective  actions.  For  details  on  the  Army  and  Air  Force  systems,  please  see 
Appendix  C. 
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Navy  Process 


Recordkeeping.  The  Navy  process  for  tracking  corrective  actions  was  not 
effective  because  the  NSC  set  a  low  priority  for  updating  the  status  of  corrective 
actions  data  in  SIMS.  SIMS,  an  Informix  dynamic  relational  database  developed 
in  1993,  was  the  primary  database  for  safety  information. 

The  NSC  Database  Management  and  Retrieval  Division  experienced  a 
70  percent  reduction  in  data  coders  over  the  last  5  years,  which  caused  an 
increased  workload  for  the  three  remaining  coders.  As  a  result,  the  SIMS 
manager  began  prioritizing  data  entries.  Data  entry  for  hazard  reports,  safety 
investigation  reports,  and  endorsement  changes  received  a  higher  priority  than 
updating  the  corrective  actions  status.  At  that  point,  data  entry  for  changing  the 
status  of  corrective  actions  became  backlogged.  NSC  aircraft  analysts  maintained 
aircraft  model  files  to  track  recommended  corrective  actions  to  closure.  The 
aircraft  analyst  submitted  data  recommending  record  closure  to  the  SIMS 
manager  when  the  action  agency  completed  the  recommended  corrective  action. 
The  SIMS  manager  could  not  determine  how  many  open  corrective  actions  in 
SIMS  were  implemented  because  data  were  backlogged  for  several  months. 

Corrective  Actions  Tracking,  The  NSC  had  significant  data  backlogs  which 
caused  data  from  SIMS  to  be  unreliable.  The  NSC  used  the  Mishap  and  Hazard 
Recommendation  Tracking  (MISTRAC)  program  to  monitor  corrective  actions 
data  entered  in  SIMS.  MISTRAC  data  included  corrective  actions  submissions  to 
eliminate  hazards.  MISTRAC  showed  339  corrective  actions  for  class  “A” 
aircraft  mishaps  in  SIMS  as  open.  Although  major  commands  had  implemented 
corrective  actions,  the  data  closing  the  case  in  SIMS  had  not  been  entered  because 
of  the  backlog.  A  backlogged  system  has  unreliable  or  outdated  data  that  could 
result  in  poor  decision  making  and  inefficient  resourcing  by  senior  management. 
For  example,  we  compared  the  SIMS  data  for  three  specific  airframes  assigned  to 
the  Naval  Air  Systems  Command  to  the  data  from  the  Naval  Air  Systems 
Command.  The  Naval  Air  Systems  Command  had  implemented  34  of  the  53  (64 
percent)  open  recommended  corrective  actions  in  SIMS.  See  the  following  table 
for  a  comparison  of  open  corrective  actions  in  SIMS  and  the  Naval  Air  Systems 
Command.  The  difference  demonstrates  that  the  corrective  actions  implemented 
by  the  Naval  Air  Systems  Command  were  not  reflected  as  closed  in  SIMS. 
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August  2000  Open  Class  “A” 
Recommended  Corrective  Actions 

■  Safety  Information  Management  Systems 
□  Naval  Air  Systems  Command 


AV-8  F-14  F-18  Totals 

Aircraft  Type 


Navy  Query  System,  The  NSC  used  the  Naval  Safety  Interactive  Retrieval 
System  (NSIRS)  to  query  data.  NSIRS  was  a  sophisticated  database  query  system 
for  advanced  computer  users.  NSIRS  provided  an  access  path  to  SIMS  and 
served  as  an  interface  for  customers  through  a  structured  query  language.  Naval 
aircraft  analysts  and  field  organizations  had  not  used  NSIRS  to  query  SIMS 
because  they  lacked  the  technical  expertise  for  this  complex  system. 

Accessibility  of  SIMS  Database,  Although  the  NSC  strategic  plan  included  a 
goal  to  find  better  ways  to  analyze  database  information,  the  NSC  was  unable  to 
meet  their  strategic  mission  for  making  safety  information  easily  accessible  to  all 
Navy  personnel.  NSC  provided  customers  with  accurate  data  on  the  status  of 
corrective  actions;  however,  the  process  required  a  duplication  of  effort  by  NSC 
aircraft  analysts.  Naval  aircraft  analysts  and  field  organizations  did  not  have  the 
technical  expertise  to  query  SIMS;  therefore,  NSC  aircraft  analysts  tracked  the 
status  of  corrective  actions  on  their  desktop  computers.  When  field  organizations 
requested  data  from  the  aircraft  analyst,  the  analyst  processed  the  request  through 
the  SIMS  manager  to  query  the  data  from  SIMS.  The  analysts  compared  the  data 
from  SIMS  to  the  data  on  their  desktop  computers  to  ensure  that  customers 
received  accurate  data. 

Reporting  Corrective  Action  Performance,  The  NSC  was  unable  to  efficiently 
report  corrective  action  performance  because  the  status  of  corrective  actions  in  the 
database  were  unreliable.  The  purpose  of  reporting  corrective  action  performance 
is  to  measure  the  effectiveness  of  mishap  prevention  efforts.  Effective 
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implementation  of  correetive  actions  was  essential  to  preventing  future  mishaps 
that  could  result  in  loss  of  equipment  or  life.  Performance  measurement  of 
corrective  actions  implementation  would  give  Navy  leaders  and  safety  managers 
the  information  that  highlighted  implementation  problems  and  provided  the 
background  for  making  informed  decisions. 

Conclusion 


The  Military  Departments  complied  with  DoD  requirements  for  aviation  mishap 
investigations,  reports,  and  recordkeeping.  Each  Military  Department  flight 
mishap  investigation  was  independent  and  identified  root  causes  of  the  mishap. 
The  aircraft  mishap  investigation  board  used  the  safety  center  analysts  and 
technical  experts  in  the  investigation  process.  Investigation  boards  reported 
aviation  mishap  investigations  in  a  timely  manner.  Each  Military  Department 
Safety  Center  relied  on  the  integrity  of  the  aircraft  mishap  investigation  board  and 
the  endorsement  chain  for  factual  accuracy  of  information. 

The  Navy  recordkeeping  process  did  not  effectively  track  the  status  of  corrective 
actions.  The  NSC  strategic  plan  included  a  goal  to  find  better  ways  to  analyze 
database  information  and  a  strategic  mission  to  make  safety  information  easily 
accessible  to  all  Navy  personnel.  The  NSC  SIMS  did  not  meet  those  objectives 
and  did  not  efficiently  report  Navy  corrective  action  performance.  The  NSC 
information  system  for  managing  aviation  mishap  corrective  actions  data  needs 
user-friendly  capability  similar  to  the  systems  used  by  the  other  Military 
Departments  and  accessible  through  the  internet.  Also,  by  establishing  a 
performance  oversight  process  for  implementing  corrective  actions.  Navy  leaders 
and  safety  managers  would  have  the  tools  needed  to  make  informed  resourcing 
and  personnel  decisions  that  could  effect  significant  reductions  in  accidents. 

Recommendations  and  Management  Comments 


We  recommend  that  the  Vice  Chief  of  Naval  Operations  direct  the  Naval 
Safety  Center  to  update  the  safety  information  management  system  to: 

1.  Track  the  timely  status  of  corrective  actions  associated  with 
aircraft  mishaps. 

Management  Comments.  The  Vice  Chief  of  Naval  Operations  concurred  and 
stated  that  they  recognize  the  weakness  in  corrective-action  tracking  and  agreed 
that  the  recordkeeping  process  was  not  effective  because  of  priorities  and  staffing. 
However,  the  Vice  Chief  of  Naval  Operations  stated  that  the  process  of  retrieving 
information  from  the  action  agencies  is  in  need  of  attention  and  not  the  SIMS 
database.  The  Naval  Safety  Center  is  either  taking  or  planning  to  take  the 
following  actions. 
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•  Hire  a  GS-12  civilian  to  oversee  the  tracking  of  corrective  actions 
listed  in  mishap  recommendations  and  hazard  reports. 

•  Assign  two  Naval  Safety  Center  Reservists  to  contact  all  action 
agencies  and  update  the  status  of  all  open  mishap  recommendations. 
This  research  will  significantly  reduce  the  number  of  open 

rec  ommendations . 

•  Send  a  list  of  open  recommendations,  twice  a  year,  to  all  action 
agencies  asking  that  they  respond  within  30  days  indicating  whether 
the  recommendation  has  been  completed  or  provide  an  update  to  its 
status. 

•  Send  a  similar  list,  twice  a  year,  to  all  controlling  custodians. 

•  Send  a  list  of  open  recommendations,  monthly,  to  the  command's 
aircraft  analysts  to  give  them  the  information  necessary  to  take  a  more 
proactive  approach  with  outside  agencies. 

2,  Simplify  system  access  for  management  and  safety  officials. 

Management  Comments.  The  Vice  Chief  of  Naval  Operations  concurred  and 
stated  that  the  SIMS  database  is  the  most  extensive  safety  database  in  the  DoD. 
Although  the  database  appears  complex  and  unwieldy,  SIMS  permits  NSC 
analysts  to  conduct  detailed  mishap  analysis  and  research  vital  information, 
striving  to  save  lives  and  resources.  The  NSC  purposely  restricted  direct  access  to 
the  database  to  protect  the  privileged  nature  of  much  of  the  data  and  to  ensure  that 
untrained  personnel  do  not  misinterpret  data. 

The  Vice  Chief  of  Naval  Operations  stated  that  two  initiatives,  Navy/Marine 
Corps  Intranet  and  Web  Enabled  Navy,  will  establish  a  conduit  for  fully 
functional  Internet  access  to  NSC  databases  for  authorized  customers  and  limited 
access  to  non-privileged  data  for  all  others. 
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Appendix  A.  Evaluation  Process 

Scope 


We  reviewed  the  Military  Serviees'  polieies  and  proeedures  that  implement  DoD 
poliey  for  aireraft  mishap  investigation  and  reporting.  We  also  reviewed  the  role 
of  the  Military  Department  Safety  Centers,  seleeted  major  eommands,  and  field 
operating  units  for  aireraft  mishap  investigations,  reports,  reeordkeeping,  and 
proeess  for  traeking  eorreetive  aetions.  We  judgmentally  seleeted  101  safety  and 
legal  mishap  investigation  reports  for  elass  “A”  aireraft  mishaps  from  FY  1995 
through  FY  2000. 

DoD  published  the  revision  to  DoDI  6055.7,  “Aeeident  Investigation,  Reporting, 
and  Reeordkeeping,”  Oetober  3,  2000,  after  our  evaluation  projeet  began. 

Aviation  mishap  investigation  requirements  generally  remained  the  same; 
therefore,  implementation  of  the  new  instruetion  was  not  ineluded  in  the 
evaluation. 

DoD-Wide  Corporate-Level  Government  Performance  and  Results  Act 
Coverage.  In  response  to  the  Government  Performanee  and  Results  Act,  the 
Secretary  of  Defense  annually  establishes  DoD-wide  corporate-level  goals, 
subordinate  performance  goals,  and  performance  measures.  This  report  pertains 
to  achievement  of  the  following  corporate-level  goal,  subordinate  performance 
goal,  and  performance  measure. 

FY  2001  DoD  Corporate-Level  Goal  2:  Prepare  now  for  an  uncertain 
future  by  pursuing  a  focused  modernization  effort  that  maintains  U.S. 
qualitative  superiority  in  key  warfighting  capabilities.  Transform  the  force 
by  exploiting  the  Revolution  in  Military  Affairs,  and  reengineer  the 
Department  to  achieve  a  21st  century  infrastructure.  (Ol-DoD-02) 

FY  2001  Subordinate  Performance  Goal  2,5:  Improve  DoD  financial 
and  information  management.  (Ol-DoD-2,5) 

FY  2001  Performance  Measure  2,5.3:  Qualitative  assessment  of 
reforming  information  technology  management.  (01— DoD-2,5,3.) 

DoD  Functional  Area  Reform  Goals,  Most  major  DoD  functional  areas  have 
also  established  performance  improvement  reform  objectives  and  goals.  This 
report  pertains  to  achievement  of  the  following  functional  area  objectives  and 
goals. 


•  Environment  Area.  Objective:  Protect  human  resources  with  the 
annual  goal  of  achieving  significant  reductions  in  all  accidents  and 
occupational  injuries  and  illnesses.  Goal:  Apply  risk  management 
techniques  to  aviation  safety,  ground  safety,  traffic  safety,  and  safety  and 
occupational  health.  (Env-5,1.1) 
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•  Information  Technology  Management  Area,  Objective:  Provide 
services  that  satisfy  customer  information  needs.  Goal:  Modernize  and 
integrate  the  Defense  Information  infrastructure,  evolving  it  to  the  Global 
Information  Grid.  (IM-2,3) 

General  Accounting  Office  High-Risk  Area,  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  coverage 
of  the  Defense  information  technology  high-risk  area. 

Methodology 


To  accomplish  the  evaluation,  we  identified,  gathered,  and  analyzed  existing 
requirements,  policy,  and  guidance  related  to  DoD  aircraft  mishap  investigations 
and  reports  by: 

•  conducting  site  visits  to  the  Military  Department  Safety  Centers  to  review 
the  investigation  and  reporting  process, 

•  conducting  site  visits  to  selected  major  commands  to  review  the  process 
for  tracking  corrective  actions, 

•  interviewing  personnel  at  selected  wings  and  squadrons  to  obtain  their 
views  on  the  effectiveness  of  the  aviation  safety  program, 

•  reviewing  safety  and  legal  reports  for  class  “A”  aircraft  mishaps  to  verify 
implementation  of  DoD  policy  and  to  compare  causal  factors,  and 

•  interviewing  officials  from  other  agencies  to  determine  their  methods  for 
aircraft  mishap  investigation  and  reporting. 

Use  of  Computer-Processed  Data,  To  achieve  the  evaluation  objectives,  we 
relied  on  computer-processed  data  contained  in  the  Army,  Navy,  and  Air  Force 
safety  information  systems.  Nothing  came  to  our  attention  as  a  result  of  specified 
procedures  that  caused  us  to  doubt  the  reliability  of  the  data  contained  in  the 
Army  and  Air  Force  safety  information  systems.  The  results  of  our  data  tests  of 
the  Naval  Safety  Information  Management  System  (SIMS)  showed  that  there 
were  open  recommendations  in  SIMS  that  had,  in  fact,  been  closed,  casting  doubt 
on  the  data's  reliability.  However,  when  the  data  are  reviewed  in  context  with 
other  available  evidence,  we  believe  that  the  opinions,  conclusions,  and 
recommendations  in  this  report  are  valid. 

Evaluation  Type,  Dates,  and  Standards,  We  performed  this  program  evaluation 
from  August  2000  through  April  2001  in  accordance  with  standards  implemented 
by  the  Inspector  General,  DoD.  Accordingly,  we  included  tests  of  management 
controls  considered  necessary. 

Contacts  During  the  Evaluation,  We  visited  or  contacted  individuals  and 
organizations  within  DoD  and  other  Federal  agencies.  Further  details  are 
available  on  request. 
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Management  Control  Program  Review 


DoD  Directive  5010.38,  “Management  Control  (MC)  Program,”  August  26,  1996, 
and  DoD  Instruction  5010.40,  “Management  Control  (MC)  Program  Procedures,” 
August  28,  1996,  require  DoD  organizations  to  implement  a  comprehensive 
system  of  management  controls  that  provides  reasonable  assurance  that  programs 
are  operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  the  Management  Control  Program.  We  reviewed  the 
adequacy  of  NSC  management  controls  over  the  implementation  of  the  DoD 
policy  for  aircraft  mishap  investigation  and  reporting.  Specifically,  we  reviewed 
NSC  management  controls  over  investigations,  reports,  recordkeeping,  and  the 
process  for  tracking  corrective  actions.  We  assessed  management’s 
self-evaluation. 

Adequacy  of  Management  Controls.  We  identified  material  management 
control  weaknesses  for  NSC  as  defined  by  DoD  Instruction  5010.40.  NSC 
management  controls  for  implementing  the  DoD  aviation  mishap  investigations, 
reports  and  recordkeeping  program  were  adequate;  however,  the  Navy  process  for 
tracking  corrective  actions  was  not  effective.  The  NSC  set  a  low  priority  for 
updating  the  status  of  corrective  actions  data  in  SIMS.  Implementation  of 
Recommendations  1.  and  2.  will  improve  NSC's  corrective  action  tracking 
process.  A  copy  of  the  report  will  be  provided  to  the  senior  official  responsible 
for  management  controls  at  the  Naval  Safety  Center. 

Adequacy  of  Management's  Self-Evaluation.  The  NSC  did  not  identify 
tracking  corrective  actions  as  an  assessable  unit  and,  therefore,  did  not  identify  or 
report  the  material  management  control  weaknesses  identified  by  the  evaluation. 
However,  NSC  plans  to  revisit,  in  their  2001  Strategic  Plan,  the  tracking  of 
aviation  mishap  corrective  actions  and  will  address  it  within  Goal  3,  Objective  1 
of  their  2001  Strategic  Plan,  “. .  .Evaluate  Process/Product  Effectiveness”  and  treat 
tracking  of  aviation  mishap  corrective  actions  as  a  separate  assessable  unit. 

Prior  Coverage 


During  the  last  5  years,  the  General  Accounting  Office  issued  two  reports 
addressing  aircraft  safety. 

GAO  Report  No.  NSIAD-98-95BR,  “Military  Aircraft  Safety,  Serious  Accidents 
Remain  at  Historically  Eow  Eevels,”  March  23,  1998. 

GAO  Report  No.  NSIAD-96-69BR,  “Military  Aircraft  Safety,  Significant 
Improvements  Since  1975,”  Eebruary  1,  1996. 
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Appendix  B.  Corrective  Actions  Closure  Process 


Different  Processes  for  Closure.  The  Military  Departments  had  different 
proeesses  for  elosing  reeommended  eorreetive  aetions.  The  Army  Safety  Center 
elosed  a  eorreetive  action  when  the  major  command  accepted  responsibility  for 
implementation.  This  process  did  not  reflect  whether  the  corrective  action  was 
fully  implemented.  The  Naval  Safety  Center  closed  a  corrective  action  after  the 
NSC  analyst  notified  the  SIMS  manager  that  the  corrective  action  was 
implemented;  however,  there  was  a  significant  backlog,  and  the  corrective  action 
data  in  SIMS  were  unreliable.  The  Air  Force  Safety  Center  updated  the  database 
and  kept  all  corrective  actions  open  in  the  tracking  system  until  fully 
implemented.  The  following  table  demonstrates  that  the  number  of 
noncumulative  open  corrective  actions  in  the  Military  Department's  databases 
varied  dramatically  because  of  the  different  processes. 


Open  Corrective  Actions  For  Class  “A”  Aircraft  Mishaps 
FYs  1981  through  2000 


Fiscal  Year 

Armv 

Navv 

Air  Force 

Total 

2000 

4 

143 

66 

213 

1999 

0 

82 

87 

169 

1998 

0 

60 

41 

101 

1997 

0 

26 

28 

54 

1996 

0 

5 

12 

17 

1995 

0 

23 

14 

37 

Total 

4 

339 

248 

591 

1994-  1981 

0 

0 

21 

21 

Total 

4 

339 

269 

612 

Implementation  of  Corrective  Actions.  The  DoDI  6055.7  requires  that  the 
Heads  of  DoD  Components  establish  procedures  to  verify  that  corrective  actions 
are  taken  on  identified  mishap  causal  factors.  The  table  shows  that  612  corrective 
actions,  recommended  by  safety  investigations  for  class  “A”  aircraft  mishaps, 
were  reflected  as  not  implemented  in  the  Military  Department  tracking  databases. 
Corrective  action  recommendations  were  made  to  correct  a  causal  factor  of  a  class 
“A”  aircraft  mishap  and  to  prevent  future  mishaps.  Corrective  actions  were 
delayed  or  not  implemented  for  various  reasons,  such  as:  insufficient  funds, 
limited  personnel,  professional  or  philosophical  disagreement  with  the 
recommended  corrective  action,  not  cost  beneficial,  long  logistical  lead  times,  and 
economic  decisions  for  modifying  equipment  which  was  nearing  the  end  of  its 
useful  life. 
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Appendix  C.  Army  and  Air  Force  Processes 


Army  Process 


Recordkeeping.  The  Army  Safety  Center  (ASC)  eonverted  the  Army  Safety 
Management  Information  System  database  to  a  new  Oraele  database 
approximately  3  years  ago.  The  ASC  made  the  Risk  Management  Information 
System  (RMIS)  available,  through  the  internet,  to  safety  offieers  (of  all  Serviees) 
worldwide.  RMIS  was  the  primary  interfaee  to  safety  data  stored  in  the  Army 
Safety  Management  Information  System.  The  ASC  provided  Diseoverer  software 
that  allowed  major  eommands  to  query  teehnieal  information  from  remote 
loeations.  In  addition,  the  ASC  also  offered  training  and  data  dictionaries  to 
major  command  safety  engineers  to  assist  in  constructing  data  queries. 

Corrective  Actions  Tracking.  The  Army  tracking  system  indicated  four  open 
corrective  actions  for  class  “A”  aircraft  mishaps.  The  Army  used  the 
Recommendation  Tracking  System,  which  was  a  data  tracking  program  in  RMIS. 
When  a  major  command  accepted  responsibility  for  implementing  a 
recommended  corrective  action,  the  ASC  changed  the  status  in  the 
Recommendation  Tracking  System  to  closed.  At  that  point,  the  major  command 
was  tasked  to  assign  the  corrective  action  to  the  appropriate  organization,  monitor 
and  track  the  progress,  and  provide  the  status  to  the  ASC.  The  ASC  Director  of 
Operations,  conducted  monthly  meetings  to  track  the  progress  of  open  corrective 
actions.  The  ASC  provided  quarterly  briefings  to  the  Army  Chief  of  Staff.  The 
quarterly  briefings  discussed  major  command  assignments  and  comments  from 
the  commands.  The  Army  Chief  of  Staff  or  Vice  Chief  of  Staff  resolved  all 
assignment  disputes  among  the  ASC  and  the  major  commands. 

Air  Force  Process 


Recordkeeping.  The  Air  Force  Safety  Center  (AFSC)  updated  and  enhanced 
their  aircraft  mishap  databases  and  reporting  processes  in  FY  1998.  The  AFSC 
used  the  Safety  Automation  System  (SAS),  a  globally  accessible  web-based 
network,  to  report  and  analyze  mishaps.  SAS  provided  secure,  easy,  and 
worldwide  access  of  Air  Force  ground  safety  data  to  safety  officers  of  all 
Services.  SAS  also  provided  ad-hoc  query  capability  to  mishap  reports  with 
simple  data  entry  forms  and  various  query  output  reports.  The  AFSC  offered 
training  and  published  user's  guides  to  assist  in  constructing  database  queries. 

The  AFSC  continued  to  enhance  the  mishap  databases  by  developing  the  aviation 
SAS.  Aviation  SAS  will  allow  users  in  the  field  to  query  aviation  safety 
information  from  remote  locations.  The  AFSC  anticipates  that  aviation  SAS  will 
go  online  by  December  2001. 
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Corrective  Actions  Tracking.  The  Air  Force  had  248  open  corrective  actions 
for  class  “A”  aircraft  mishaps;  however,  corrective  actions  remained  open  in  the 
Air  Force  tracking  system  until  the  recommendation  was  fully  implemented.  The 
AFSC  tasked  the  appropriate  major  command  to  verify  that  the  corrective  actions 
were  implemented  in  the  final  mishap  report.  The  major  command's  formal 
corrective  action  review  process  provided  minutes  to  the  AFSC  on  the  status  of 
corrective  actions  semiannually.  When  the  major  command  confirmed  that 
corrective  actions  were  completed,  the  AFSC  changed  the  status  to  closed.  Not 
only  did  the  AFSC  brief  the  Air  Force  Chief  of  Staff,  but  safety  issues  were 
discussed  at  meetings  of  all  senior  Air  Force  leadership,  including  selected 
Corona  meetings.  The  Corona,  hosted  by  the  Air  Force  Chief  of  Staff,  met  three 
times  a  year  to  address  issues,  share  information,  and  formulate  policy. 
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Appendix  D.  Report  Distribution 


Office  of  the  Secretary  of  Defense 
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Deputy  Under  Secretary  of  Defense  (Installations  and  Environment) 
Under  Secretary  of  Defense  (Comptroller/Chief  Financial  Officer) 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Army 
Commander,  Army  Safety  Center 


Department  of  the  Navy 

Vice  Chief  of  Naval  Operations 
Naval  Inspector  General 
Auditor  General,  Department  of  the  Navy 
Commander,  Naval  Safety  Center 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 
Commander,  Air  Force  Safety  Center 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subeommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Serviees 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subeommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Serviees 
House  Committee  on  Government  Reform 

House  Subeommittee  on  Government  Effieieney,  Finaneial  Management,  and 
Intergovernmental  Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  National  Seeurity,  Veterans  Affairs,  and  International  Relations, 
Committee  on  Government  Reform 

House  Subeommittee  on  Technology  and  Procurement  Policy,  Committee  on 
Government  Reform 
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Department  of  Navy  Comments 


DEPARTMENT  OF  THE  NAVY 

OFFICE  OF  THE  CHIEF  OF  NAVAL  OPERATIONS 
2000  NAVY  PENTAGON 
WASHINGTON.  D.C.  20350-2000 

IN  REPLY  REFER  TO 

3  August  2001 

From:  Vice  Chief  of  Naval  Operations 

To:  Department  of  Defense,  Inspector  General 

Subj :  RESPONSE  TO  DRAFT  EVALUATION  REPORT 

Ref:  (a)  Draft  of  proposed  DoD  IG  Evaluation  Report,  Project 

No.  D2000CB-0236.000  of  11  May  2001 

(b)  DoD  Directive  7650.3 

1.  In  response  to  reference  (a),  the  following  comments  are 
provided  per  reference  (b) . 

a.  Recommendation.  "We  recommend  that  the  VCNO  direct  the 
NSC  to  update  the  Safety  Information  Management  System  (SIMS) 
to:  (1)  Trac)c  the  timely  status  of  corrective  actions  associated 
with  aircraft  mishaps,  and  (2)  Simplify  system  access  for 
management  and  safety  officials." 

b.  Discussion. 


(1)  Recommendation  1.  Concur.  The  Naval  Safety  Center 
recognizes  the  weakness  in  corrective-action  tracking  and  agrees 
that  the  recordkeeping  process  was  not  effective  because  of 
priorities  and  staffing.  However,  it  is  the  process  of 
retrieving  information  from  the  action  agencies  and  not  the  SIMS 
database  that  is  in  need  of  attention.  The  Naval  Safety  Center 
is  either  taking  or  planning  the  following  actions: 

(a)  Hiring  a  GS-12  civilian  to  oversee  the  tracking 
of  corrective  actions  listed  in  mishap  recommendations  and 
hazard  reports. 

(b)  Assigned  two  Naval  Safety  Center  Reservists  to 
contact  all  action  agencies  and  update  the  status  of  all  open 
mishap  recommendations.  This  research  will  significantly  reduce 
the  number  of  open  recommendations. 

(c)  Twice  per  year.  Commander,  Naval  Safety  Center 
will  send  a  list  of  open  recommendations  to  all  action  agencies 
asking  that  they  respond  within  30  days  indicating  the 
recommendation  has  been  completed  or  provide  an  update  to  its 
status.  These  reports  will  be  forwarded  on  1  March  and  1 
September. 

(d)  Additionally,  on  1  June  and  1  December  Commander 
Naval  Safety  Center  will  send  a  similar  list  to  all  controlling 
custodians . 


Subj:  RESPONSE  TO  DRAFT  EVALUATION  REPORT 

(e)  Once  a  month,  the  Safety  Center's  data  retrieval 
division  will  send  a  list  of  open  recommendations  to  the 
command's  aircraft  analysts.  This  process  improvement  gives 
analysts  the  information  necessary  to  take  a  more  proactive 
approach  with  outside  agencies. 

The  analysts  currently  make  every  effort  to  close 
recommendations  when  they  write  Class  "A"  final  endorsements. 
Those  that  remain  open  usually  require  extensive  engineering 
research,  funding,  and  procurement  and  installation  planning. 
Keeping  these  recommendations  "open"  until  confirmation  is 
received  ensures  that  no  corrective  action  is  prematurely 
closed. 

(2)  Recommendation  2.  Concur.  The  SIMS  database  is  the 
most  extensive  safety  database  in  the  Department  of  Defense. 
While  the  vast  amount  of  data  it  includes  may  make  the  database 
appear  complex  and  unwieldy,  SIMS  lets  Naval  Safety  Center 
analysts  conduct  detailed  mishap  analysis  and  research — vital 
information  when  striving  to  save  lives  and  resources.  Direct 
access  to  the  database  is  purposely  restricted  for  two  reasons: 
to  protect  the  privileged  nature  of  much  of  the  data  and  to 
ensure  that  untrained  personnel  don't  misinterpret  data. 

Providing  careful  analysis  and  useful  information  about 
mishap  causal  factors  has  been,  and  will  continue  to  be,  one  of 
the  most  important  functions  of  the  Naval  Safety  Center.  The 
Navy /Marine  Corps  Intranet  (NMCI)  and  Web  Enabled  Navy  (WEN) 
initiatives  will  establish  a  conduit  for  fully  functional 
Internet  access  to  NSC  databases  for  authorized  customers  and 
limited  access  to  non-privileged  data  for  all  others. 

2.  As  always,  I  appreciate  your  efforts  in  helping  identify 
ways  in  which  we  can  improve  the  safety  posture  of  the  Navy.  I 
am  confident  the  Naval  Safety  Center  is  on  the  right  track  with 
regard  to  the  above  recommendations.  If  you  have  further 
questions  or  recommendations,  please  feel  free  to  contact 
Commander,  Naval  Safety  Center 


Vice  Chief  of  Naval  Operations 
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Evaluation  Team  Members 


The  Contract  Management  Directorate,  Office  of  the  Assistant  Inspector  General 
for  Auditing,  DoD  prepared  this  report.  Personnel  of  the  Office  of  the  Inspector 
General,  DoD,  who  contributed  to  the  report  are  listed  below. 

Richard  B.  Jolliffe 
William  C.  Gallagher 
Michael  R.  Herbaugh 
Marcia  L.  Kilby 
Robert  M.  Sacks 
Joseph  M.  Kaseler 
Scott  G.  Stadtmueller 


